Papers with personality tests
InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews (2024.acl-long)
Copied to clipboard
Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, Yanghua Xiao
| Challenge: | Existing methods focus on knowledge and linguistic patterns of characters. |
| Approach: | They propose to evaluate character fidelity of role-playing agents with psychological scales . they propose to use psychological scale to measure personality traits of RPAs based on personality traits. |
| Outcome: | The proposed model reproduces character fidelity with psychological scales and shows that it is effective in measuring personality traits. |
Evaluating Psychological Safety of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | a recent study evaluated the psychological safety of large language models. |
| Approach: | They designed unbiased prompts to evaluate the psychological safety of large language models. |
| Outcome: | The proposed prompts showed that they were fine-tuned with behavioral metrics to reduce toxicity. |
On the Reliability of Psychological Scales on Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on examining Large Language Models’ characteristics from a psychological standpoint, acknowledging the necessity of understanding their behavioral characteristics. |
| Approach: | They propose to examine the reliability of personality tests to LLMs by using psychological scales. |
| Outcome: | The proposed model can represent diverse personalities with specific prompt instructions. |
Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics (2025.findings-naacl)
Copied to clipboard
Seungbeen Lee, Seungwon Lim, Seungju Han, Giyeong Oh, Hyungjoo Chae, Jiwan Chung, Minju Kim, Beong-woo Kwak, Yeonsoo Lee, Dongha Lee, Jinyoung Yeo, Youngjae Yu
| Challenge: | Recent advances in Large Language Models (LLMs) have led to their adaptation as conversational agents. |
| Approach: | They propose a new benchmark that uses 8K multi-choice questions to assess the personality of Large Language Models. |
| Outcome: | The proposed personality test outperforms existing personality tests for LLMs in reliability and validity. |